Artificial neural networks have utility in a wide variety of computing environments, such as speech recognition, process control, optical character recognition, signal processing, and image processing. Processing engines for many of the foregoing may be implemented through neural networks comprising a plurality of elemental logic elements called neuron circuits.
A neuron circuit is the fundamental building block of a neural network. A neuron circuit has multiple inputs and one output. As described in the Related Invention No. 1 identified above, the structure of a conventional neuron circuit often includes a multiplier circuit, a summing circuit, a circuit for performing a non-linear function (such as a binary threshold or sigmoid function), and circuitry functioning as synapses or weighted input connections. The above-identified Related Invention No. 1 discloses, in one embodiment, a neuron circuit which comprises only an adder as its main processing element.
As discussed in Related Invention No. 2 identified above, there are more than twenty known types of neural network architectures, of which the "back-propagation", "perceptron", and "Hopfield network" are the best known. Related Invention No. 2 discloses, in one embodiment, a neural network which does not require repetitive training, which yields a global minimum to each given set of input vectors, and which has an architecture that is easy to implement.
As is known, conventional processing engines (yon Neumann type) comprise two fundamental blocks, a computing unit and a memory unit from which the computing unit accesses instructions and data to sequentially execute its functions. The only method available to increase the computing power thereof is to increase the computing speed, and thus through an increase in the number of operations executable per unit of time.
FIG. 1 shows a prior art yon Neumann type processing engine. To perform a function in general, CPU 1 (central processing unit) repetitively executes the following sequential steps which form the basis of operation for any yon Neumann computer. First, CPU 1 retrieves an instruction from main memory 3 via CPU Bus 4. Next, CPU 1 fetches data from main memory 3 and performs an arithmetic or logical operation on the data according to the aforesaid instruction. Finally, CPU 1 stores the result of the executed operation in main memory 3.
The primary limitation of the yon Neumann computer is that it performs only one operation at a time, and although the operation may occur in a very short time, the mere fact that operations are executed serially precludes a dramatic increase in computational speed. As a result, computer architectures have been conceived that are capable of executing operations concurrently.
FIG. 2 shows a prior art multiple-processor computer. The multiple-processor computer comprises a plurality of yon Neumann computers 7, 9, 11 that communicate with each other via an interconnection network 14. By the fact that it can execute a multitude of operations at one time, the multiple-processor computer affords increased computing power over the traditional yon Neumann computer. However, the multiple-processor computer is expensive and difficult both to implement and to program. Executing software applications on a multiple-processor computer requires a sophisticated compiler. In addition, communication between processors frequently results in "communication bottlenecks" which degrade the overall computing power of the multiple-processor computer.
FIG. 3 shows a prior art yon Neumann computer with a math co-processor. CPU 20 is connected to math co-processor 22 and main memory 24 via CPU bus 26. CPU 20 generally performs the same functions as the above-described CPU 1 of FIG. 1. But, in addition, CPU 20 controls the operation of and data transfer to math co-processor 22. Math co-processor 20 is a logic circuit that is specially designed to perform mathematical computations in significantly less time than CPU 20. Typically, a math co-processor comprises specialized electronic circuitry to perform arithmetic operations such as floating point division and multiplication, and transcendental functions such as sine, cosine, tangent, etc. Even though a math co-processor may increase Computing power, the architecture depicted by FIG. 3 suffers from the fundamental limitation of being able to execute only one operation at a time. In addition, the math functions that are accelerated by the co-processor are predetermined by the circuitry of the math co-processor and are thus limited in number and not software re-configurable.
The above described prior art configurations provide satisfactory computing power under some, but not all, circumstances. In situations requiring intensive mathematical computation, prior art solutions generally fail to deliver adequate computing performance at a reasonable price. Thus, there is a significant need for a computing device which performs a wide variety of complicated math functions, which executes a large number operations per unit time, and which is easy to program and inexpensive to implement.